Role-based perception filter

ABSTRACT

Systems and methods for filtering objects of interest associated with an investigation. One method includes receiving a role identifier. The method includes retrieving metadata corresponding to the plurality of objects of interest. The method includes determining, with a classifier, a plurality of relationships between at least two of the objects of interest based on the metadata. The method includes identifying, based on the metadata, a subset of the plurality of relationships that are associated with the role identifier, the subset of the plurality of relationships including a subset of the plurality of objects of interest. The method includes generating a graphical representation including a first indication of the subset of the objects of interest and a second indication of the subset of the plurality of relationships. The method includes presenting the graphical representation on a display communicatively coupled to the electronic processor.

BACKGROUND OF THE INVENTION

Law enforcement and other government personnel perform investigations inthe course of their duties. For example, a police detective mayinvestigate a crime, and a building inspector may inspect a building forcompliance with applicable building codes. Some personnel may havespecific investigatory roles, each specializing in a particular area ofinvestigation. Personnel performing such investigations collectinformation on objects of interest associated with the investigation.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention, and explainvarious principles and advantages of those embodiments.

FIG. 1 is a diagram of a system for filtering objects of interest inaccordance with some embodiments.

FIG. 2 is a diagram of a server of the system of FIG. 1 in accordancewith some embodiments.

FIG. 3 is a flowchart of a method for filtering objects of interest inaccordance with some embodiments.

FIG. 4 is an example image produced by the system of FIG. 1 inaccordance with some embodiments.

FIG. 5 is a flowchart of a method for filtering a plurality of objectsof interest associated with an incident scene in accordance with someembodiments.

FIG. 6 is an example image produced by the system of FIG. 1 inaccordance with some embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

As noted, law enforcement and other government personnel performinginvestigations gather large amounts of data and documentation pertainingto the investigations. Investigatory personnel also produce reports, andgather or data relating to the investigation from government and othersources. Included in such data and documentation are many objects ofinterest related to the investigation. An object of interest is anyoneor anything identified by an investigator as being involved in orrelevant to the investigation of the incident. Objects of interest aresometimes identified in or extracted from still images and video, whichmay be collected from vehicle-mounted or body-worn cameras or from othersources.

Also included in the collected data and documentation are relationshipsbetween one or more of the objects of interest. Relationships betweenobjects of interest may be manually defined (for example, byinvestigators) or may be derived automatically (for example, using oneor more classifiers). The data and documentation may relate to variousobjects of interest, including people, vehicles, buildings, or portionsof buildings, and fixed or movable physical objects involved orpotentially involved with the investigation. In some instances, objectsof interest are identified in images of incidents related to theinvestigation (for example, an image of a crime scene). Objects ofinterest may also include digital representations of intangible data ordocuments related to the investigation. Relationships between theobjects of interest, beyond their shared association with theinvestigation, may also be useful to investigatory personnel.

Objects are related when they share a common characteristic or property.Objects of interest are also considered related when an investigatorindicates that they are. Personnel performing investigations may havedifferent investigatory roles. A single investigation may includehundreds of objects or interest, each potentially having one or morerelationships to one or more other of the objects of interest. Analysisof the objects of interest and known and derived relationships mayproduce an overwhelming amount of data. This may hinder rather than helpan investigation. Accordingly, systems and methods are provided hereinfor, among other things, filtering objects of interest and therelationships between them using a role-based perception filter.

One example embodiment provides a system for filtering objects ofinterest associated with an investigation. The system includes adatabase, a display, and an electronic processor communicatively coupledto the database and the display. The electronic processor is configuredto receive a role identifier. The electronic processor is configured toretrieve, from the database, metadata corresponding to the plurality ofobjects of interest. The electronic processor is configured todetermine, with a classifier, a plurality of relationships between atleast two of the objects of interest based on the metadata. Theelectronic processor is configured to identify, based on the metadata, asubset of the plurality of relationships that are associated with therole identifier, the subset of the plurality of relationships includinga subset of the plurality of objects of interest. The electronicprocessor is configured to generate a graphical representation includinga first indication of the subset of the objects of interest and a secondindication of the subset of the plurality of relationships. Theelectronic processor is configured to present the graphicalrepresentation on the display.

Another example embodiment provides a method for filtering objects ofinterest associated with an investigation. The method includesreceiving, with an electronic processor, a role identifier. The methodincludes retrieving, from a database communicatively coupled to theelectronic processor, metadata corresponding to the plurality of objectsof interest. The method includes determining, with a classifier, aplurality of relationships between at least two of the objects ofinterest based on the metadata. The method includes identifying, basedon the metadata, a subset of the plurality of relationships that areassociated with the role identifier, the subset of the plurality ofrelationships including a subset of the plurality of objects ofinterest. The method includes generating a graphical representationincluding a first indication of the subset of the objects of interestand a second indication of the subset of the plurality of relationships.The method includes presenting the graphical representation on a displaycommunicatively coupled to the electronic processor.

Another example embodiment provides a method for filtering a pluralityof objects of interest. The method includes receiving, with anelectronic processor, a plurality of object identifiers, each of theobject identifiers corresponding to one of the plurality of objects ofinterest associated with an image of the incident scene. The methodincludes receiving metadata for the plurality of object identifiers. Themethod includes determining, with the electronic processor, a pluralityof relationships based on the metadata, where each of the plurality ofrelationships corresponds to at least two of the plurality of objects ofinterest. The method includes generating a graphical representationbased on the plurality of objects of interest and the plurality ofrelationships. The graphical representation includes a plurality ofnodes, each node providing an indication of one of the plurality ofobjects of interest. The graphical representation also includes aplurality of edges, each edge connecting two of the plurality of nodesand providing an indication of one of the plurality of relationships.The method includes receiving a role identifier. The method includes, inresponse to receiving the role identifier, applying a role-basedperception filter corresponding to the role identifier to the pluralityof objects of interest and the plurality of relationships to generate afiltered graphical representation. The method includes presenting thefiltered graphical representation on a display.

For ease of description, some or all of the example systems presentedherein are illustrated with a single exemplar of each of its componentparts. Some examples may not describe or illustrate all components ofthe systems. Other example embodiments may include more or fewer of eachof the illustrated components, may combine some components, or mayinclude additional or alternative components.

FIG. 1 illustrates an example system 100 for filtering objects ofinterest. In the example illustrated, the system 100 includes a server102 and a database 104. The server 102, described more particularlybelow with respect to FIG. 2, is communicatively coupled to, and writesdata to and from, the database 104. As illustrated in FIG. 1, thedatabase 104 may be a database housed on a suitable database servercommunicatively coupled to and accessible by the server 102. Inalternative embodiments, the database 104 may be part of a cloud-baseddatabase system external to the system 100 and accessible by the server102 over one or more additional networks. In some embodiments, all orpart of the database 104 may be locally stored on the server 102. Insome embodiments, as described below, the database 104 electronicallystores data on investigations, objects of interest (for example, a firstobject of interest 112 and a second object of interest 114), metadata(for example, related to the objects of interest), investigatory roles,and relationships (for example, between the objects of interest). Insome embodiments, the server 102 and the database 104 are part of acomputer-aided dispatch system.

The server 102 is communicatively coupled to an image capture device 106and a computing device 108 via a communications network 110. Thecommunications network 110 is a communications network includingwireless and wired connections. The communications network 110 may beimplemented using a local area network, such as a Bluetooth™ network orWi-Fi, a wide area network, such as the Internet, and other networks orcombinations of networks including a Long Term Evolution (LTE) network,a Global System for Mobile Communications (or Groupe Special Mobile(GSM)) network, a Code Division Multiple Access (CDMA) network, anEvolution-Data Optimized (EV-DO) network, an Enhanced Data Rates for GSMEvolution (EDGE) network, a 3G network, and a 4G network. Derivatives ofsuch networks and other suitable networks, including future-developednetworks may also be used.

The image capture device 106 is an electronic device that captures, forexample, still images, video, or both of the first object of interest112, the second object of interest 114, or other objects of interest(not shown) in the incident scene 116. As used herein, the term “image”encompasses both still images and video. In some embodiments, the imagecapture device 106 is a body-worn camera. In other embodiments, theimage capture device may be or include a smart telephone, a vehicle dashcamera, a surveillance camera, a traffic camera, or another suitableimage capture device that records video or still images or objects ofinterest. In some embodiments, the image capture device 106 transmits,via the communications network 110, the captured images and videostreams to the server 102, which extracts objects of interest from them.In some embodiments, the image capture device 106 extracts the objectsof interest from the captured images and transmits the objects to theserver 102. In some embodiments, the server 102 and the image capturedevice 106 include software and hardware to electronically detect andclassify objects within captured images and video streams (for example,video processors and object classifier algorithms). Objectclassification is known in the art, and will not be described in detailherein.

The server 102 may also receive objects of interest, and other data asdescribed below, from a computing device 108. In some embodiments, thecomputing device 108 is a computer terminal, for example, at a CommandCenter. In other embodiments, the computing device 108 is a suitableelectronic device for sending and receiving data to and from the server102 (via the communications network 110), for example, a smarttelephone, a tablet computer, a laptop computer, a smart watch, and thelike.

FIG. 2 illustrates an example of the server 102. In the embodimentillustrated, the server 102 includes an electronic processor 205, amemory 210, a communication interface 215, and a display 220. Theillustrated components, along with other various modules and componentsare coupled to each other by or through one or more control or databuses that enable communication therebetween.

The electronic processor 205 obtains and provides information (forexample, from the memory 210 and/or the communication interface 215),and processes the information by executing one or more softwareinstructions or modules, capable of being stored, for example, in arandom access memory (“RAM”) area of the memory 210 or a read onlymemory (“ROM”) of the memory 210 or another non-transitory computerreadable medium (not shown). The software can include firmware, one ormore applications, program data, filters, rules, one or more programmodules, and other executable instructions. The electronic processor 205is configured to retrieve from the memory 210 and execute, among otherthings, software related to the control processes and methods describedherein.

The memory 210 can include one or more non-transitory computer-readablemedia, and includes a program storage area and a data storage area. Theprogram storage area and the data storage area can include combinationsof different types of memory, as described herein. In the embodimentillustrated, the memory 210 stores, among other things, a classifier230. In some embodiments, the memory 210 stores and the electronicprocessor 205 executes multiple classifiers. In some embodiments, theclassifier is a machine learning algorithm (for example, a neuralnetwork or Bayes classifier).

The communication interface 215 may include a wireless transmitter ortransceiver for wirelessly communicating over the communications network110. Alternatively or in addition to a wireless transmitter ortransceiver, the communication interface 215 may include a port forreceiving a cable, such as an Ethernet cable, for communicating over thecommunications network 110 or a dedicated wired connection. It should beunderstood that, in some embodiments, the server 102 communicates withthe image capture device 106, the computing device 108, or both throughone or more intermediary devices, such as routers, gateways, relays, andthe like.

The display 220 is a suitable display such as, for example, a liquidcrystal display (LCD) touch screen, or an organic light-emitting diode(OLED) touch screen. In some embodiments, the display is integrated withthe server 102. The server 102 implements a graphical user interface(GUI) (for example, generated by the electronic processor 205, frominstructions and data stored in the memory 210, and presented on thedisplay 220), that enables a user to interact with the server 102. Insome embodiments, the server 102 operates or is integrated with ahead-mounted display (HMD), an optical head-mounted display (OHMD), orthe display of a pair of smart glasses. In some embodiments, a userinteracts with the server 102 remotely using a remote electronic device(for example, the computing device 108).

In some embodiments, the server 102 operates using, among other things,augmented reality technology, where live images are displayed (forexample, on the display 220) with text, graphics, or graphical userinterface elements superimposed on or otherwise combined with the liveimages. In some embodiments, the server 102 operates using, among otherthings, virtual reality technology, where actual or simulated images aredisplayed (for example, on the display 220) with text, graphics, orgraphical user interface elements superimposed on or otherwise combinedwith the images.

In some embodiments, the electronic processor 205 performs machinelearning functions. Machine learning generally refers to the ability ofa computer program to learn without being explicitly programmed. In someembodiments, a computer program (for example, a learning engine) isconfigured to construct an algorithm based on inputs. Supervisedlearning involves presenting a computer program with example inputs andtheir desired outputs. The computer program is configured to learn ageneral rule that maps the inputs to the outputs from the training datait receives. Example machine learning engines include decision treelearning, association rule learning, artificial neural networks,classifiers, inductive logic programming, support vector machines,clustering, Bayesian networks, reinforcement learning, representationlearning, similarity and metric learning, sparse dictionary learning,and genetic algorithms. Using one or more of these approaches, acomputer program can ingest, parse, and understand data andprogressively refine algorithms for data analytics.

As noted above, the server 102 receives objects of interest and otherdata (for example, from the database 104, the image capture device 106,and the computing device 108). As described in detail below, theelectronic processor 205 is configured to analyze and filter the objectsof interest to enhance investigations. As used herein, the term“investigation” may refer to one or more related investigative efforts.For example, an investigation may be the investigation of an incident(for example, a crime). In another example, an investigation includesthe surveillance or patrol duties performed by a police officer. Inanother example, an investigation is the response to a call for service,for example, responding to a crime, responding to a traffic accident,searching for a suspect, locating a missing person, responding to afire, responding to a medical emergency, and the like. In yet anotherexample, an investigation is an inspection of a building or otherlocation for compliance with applicable codes and regulations. Aninvestigation may include multiple related incidents or responses. Forexample, an investigation of a crime may include the initialinvestigation of the crime scene, as well as subsequent suspect andwitness interviews. In another example, an investigation includes theinvestigations of multiple crime or potential crime scenes, or otherareas of interest.

Performing an investigation includes the detection, categorization, andanalysis of objects of interest. Some personnel performinginvestigations may have specific investigatory roles. An investigatoryrole implies a specialized knowledge in a particular area ofinvestigation, for example, homicide, burglary, drugs, explosives, humantrafficking, domestic violence, code enforcement, and the like. Asdescribed below, investigatory roles may be used to characterize objectsof interest and relationships between objects of interest.

An object of interest (for example, a first object of interest 112 and asecond object of interest 114) is anyone or anything identified by aninvestigator as being involved in or relevant to an investigation.Objects of interest may also be identified automatically, for example,through the use of a classifier. Objects of interest may include, forexample, people (for example, victims or suspects), automobiles,weapons, buildings, or portions of buildings. Objects of interest mayalso be tangible things not commonly thought of as objects, but whichare still relevant to the investigation, for example, fluids leaked fromautomobiles, debris from damaged property, and blood. An object ofinterest may also be a physical property or characteristic of anotherobject (for example, a dent in an automobile, a graffiti tag on abuilding, or an article of clothing on a person). Objects of interestalso include digital representations of data or documents related to theinvestigation, for example, police reports, search warrants, records oftransactions, government documents, scanned papers, records of objectsseized during a search, sounds, detectable smells (for example, ofdrugs, chemicals, or explosives), and the like.

As noted above, one or more objects of interest may share one or morerelationships with one another. Objects are related when they share acommon characteristic. For example, a wheel may be related to a nearbyvehicle, a group of persons may be related by their association with acriminal enterprise, and an article of clothing may be related to thewearer. In another example, a group of objects that could be used tomake an explosive device are related to one another. In yet anotherexample, the components needed to make an illicit drug are related toone another.

Whether or what sort of relationship exists between objects of interestmay change with investigatory roles. One set of objects of interest maybe related one way in light of one investigatory role, but may berelated in another way, or viewed as unrelated, in light of anotherinvestigatory role. For example, a collection of household chemicals maybe related to drug production when viewed by a drug investigator, whilethe same chemicals may be related to poisoning when viewed by a homicideinvestigator, and still other investigators may see no relationships atall between the chemicals.

Some investigations involve a large amount of objects of interest andpotential relationships. Graphically presenting all of the objects ofinterest and relationships to an investigator results in a large,complex display, which may be visually confusing and difficult tonavigate. Because some of the objects of interest and relationships maybe relevant to some investigatory roles, but not to others, methods areprovided herein to filter the objects of interest and relationshipsusing role-based perception filters. For example, FIG. 3 illustrates anexample method 300 for filtering a plurality of objects of interestassociated with an investigation. The method 300 is described as beingperformed by the server 102 and, in particular, the electronic processor205. However, it should be understood that in some embodiments, portionsof the method 300 may be performed external to the server 102 by otherdevices, including for example, the image capture device 106 and thecomputing device 108.

As an example, the method 300 is described in terms of the server 102filtering a predetermined plurality of objects of interest. In someembodiments, the plurality of objects of interest may be retrieved fromthe database 104. In some embodiments, some of the plurality of objectsof interest may be received from the image capture device 106, thecomputing device 108, another source external to the server 102, or somecombination of the foregoing. Regardless of the source, each of theplurality of objects is associated with an investigation, for example,the investigation of a crime scene.

At block 302, the electronic processor 205 receives a role identifier.The role identifier corresponds to an investigatory role. In someembodiments, the electronic processor 205 receives the role identifierfrom a user operating the server 102 (for example, from a user profile).In some embodiments, the role identifier is specified by a user, but maynot correspond to the user. In some embodiments, electronic processor205 retrieves the role identifier from the database 104, for example,based on an investigator assigned to an investigation. In someembodiments, the electronic processor 205 receives multiple roleidentifiers. For example, a user of the server 102 may have an assignedrole identifier, and the same user may specify one or more additionalrole identifiers.

In some embodiments, the role identifier and corresponding investigatoryroles are stored in the database 104. In one example, each investigatoryrole has a record or records in the database identified by the roleidentifier. Each record includes information regarding the identifiedinvestigatory role, for example, what types of objects of interest maybe relevant to the role, one or more potential relationships betweentypes of objects of interest that are relevant to the role, groups ofrelated objects of interest relevant to the role, and the like.

At block 304, the electronic processor 205 retrieves, from the database,metadata corresponding to the plurality of objects of interest. Metadatais data relating to or describing the objects of interest. In someembodiments, metadata includes a timestamp for object of interestidentifying, for example, when the object was first discovered byinvestigators, when the object was cataloged, and the like. In someembodiments, metadata includes a location for the object identifying,for example, where the object was located when it was first discoveredby investigators, the current location of the object, and the like. Insome embodiments, metadata includes an image of the object, for example,extracted from an image captured by the image capture device 106. Insome embodiments, metadata includes an object identifier, for example, aglobally-unique identifier used to identify the object within thedatabase 104. In some embodiments, metadata includes an investigationidentifier, for example, a unique identifier associating the object withan investigation. In some embodiments, the unique identifier is linkedto a record in a computer-aided dispatch system, a records managementsystem, or a content management system (for example, the CommandCentralVault™ by Motorola Solutions®). In some embodiments, metadata includesan object type that characterizes the object (for example, person,automobile, weapon, document, and the like). In some embodiments,metadata includes an incident type, for example, an indicator of thetype of incident with which the object is associated (for example, acrime, a missing person, a traffic accident, and the like). In someembodiments, metadata includes a role identifier, as described above. Insome embodiments, metadata includes a potential or actual relationshipto another object of interest, for example, an automobile may be relatedto a registered owner record, or a weapon may be related to a body.

The foregoing examples should not be considered limiting. Metadatacorresponding to an object of interest is any data relating to ordescribing the object of interest or the object's relationship to otherobjects of interest, or any aspect of the object of interest that may berelevant to the investigation. Metadata may be entered by investigatorscollecting information, for example, via a police or inspection report.Metadata may also be collected automatically. In some embodiments, theelectronic processor 205 retrieves metadata related to objects ofinterest from various government and public databases. For example, anidentification number for a vehicle may be used to retrieve ownershipinformation for the vehicle. In another example, financial informationfor a suspect may be used to retrieve data on recent transactions bythat suspect from financial institution databases.

At block 306, the electronic processor 205 determines, with a classifier(for example, the classifier 230), a plurality of relationships betweenat least two of the objects of interest based on the metadata. Theclassifier 230 automatically identifies relationships between theobjects of interest based on, for example, historical determinations ofrelationships, defined relationships entered by users of the system, andmatches between the metadata for the objects of interest. For example,the electronic processor 205 may determine a relationship between twoobjects of interest when there is a property or characteristic common toboth objects of interest. In some embodiments, the electronic processor205 retrieves one or more relationships the database 104 or from userinputs. In some embodiments, the electronic processor 205 determines apotential relationship between two objects, which may be verified, forexample, by a user of the server 102. In some embodiments, theelectronic processor 205 utilizes more than one classifier toautomatically identify relationships between the objects of interest.Each classifier is associated with a role, and each role may beassociated with multiple classifiers. In some embodiments, theclassifier(s) insert a role identifier into the relationships' metadata,when the individual role-based classifiers create the relationships.

At block 308, the electronic processor 205 identifies, based on themetadata, a subset of the plurality of relationships that are associatedwith the role identifier. Each of the relationships in the subset areassociated with a common role identifier, indicating, for example, thatthe relationship was created by a classifier associated with that role.For example, the electronic processor 205 may identify a subset of theplurality of relationships that are all related to homicideinvestigations. In another example, the electronic processor 205 mayidentify a subset of the plurality of relationships that are all relatedto drug investigations. When more than one role identifier is specified(at block 302), a subset is identified for each role identifier.

Relationships between objects of interest may be identified with morethan one investigatory role. For example, a relationship between abloodstain and a weapon may be of interest to a homicide investigatorand a domestic violence expert. In another example, certain flammableliquids may be of interest to an arson investigator, because the liquidscan be used to set fires, and to a drug enforcement agent, because theymay be used in the manufacture of certain illegal drugs. In some cases,a relationship between objects of interest may be more relevant to onerole than another. For example, during a drug trafficking investigation,the aforementioned flammable liquids may be more relevant to the drugenforcement agent role than the arson investigator role. Accordingly, insome embodiments, the electronic processor 205 assigns to each of theplurality of relationships a relationship relevancy score based on themetadata for the objects of interest related by the relationship and therole identifier for the relationship. The relationship relevancy scoreis a measure (for example, a percentage) indicating the degree to whichthe relationship is relevant to a particular investigatory role.Metadata that tends to make the relationship more relevant to aninvestigatory role will increase the relationship relevancy score,whereas metadata that tends to make the object of interest less relevantto an investigatory role will decrease the relationship relevancy score.The relationship relevancy score may also be affected by how stronglythe objects are related, as determined by the classifier. In suchembodiments, the electronic processor 205 identifies the subset of theplurality of relationships based on the relationship relevancy score. Insome embodiments, the electronic processor 205 determines whether arelationship is included in the subset by comparing the relationshiprelevancy score to a threshold. In one example, the threshold is anabsolute threshold (for example, an object relevancy score of at least0.6 when the possible range is 0 to 1). In another example, thethreshold is a relative value indicative of how much higher onerelationship relevancy score is from the next nearest relationshiprelevancy score for the same role, or above a median or averagerelationship relevancy score for relationships having relationshiprelevancy scores for the same role.

Optionally, in some embodiments, the electronic processor 205 assigns aweight to one or more of the relationship relevancy scores. The weightis used to indicate how significant a particular relationship relevancyscore is to identifying a relationship with an investigatory rolerelative to other relationships. In some embodiments, the electronicprocessor 205 assigns a weight based on the metadata for the relatedobjects of interest. In some embodiments, the electronic processor 205may determine the weights using a machine learning algorithm (forexample, a neural network or Bayes classifier). Regardless of how theweight is determined, the electronic processor 205 assigns, for each ofthe relationships, a weight to the relationship relevancy score togenerate a weighted relationship relevancy score. In such embodiments,the electronic processor 205 identifies the subset of the plurality ofrelationships based on the weighted relationship relevancy score.

The subset of relationships includes a subset of the plurality ofobjects of interest. The subset of the plurality of objects of interestincludes those objects that are related by the relationships generatedby the classifier(s) that are associated with the role identifier usedto determine the subset or relationships.

In some embodiments, the subsets of relationships are identified using arole-based perception filter associated with the role identifier. Eachinvestigatory role has a corresponding role-based perception filter (forexample, stored in the database 104). A role-based perception filterstores information used to determine when an object of interest or arelationship is relevant to its role. For example, a role-basedperception filter may include examples of metadata and metadata valuesthat, when present for objects in relationship, make that relationshiprelevant to the role. A role-based perception filter may also includehistorical information indicative of what relationships have beenselected or determined in the past as relevant to its role. Role-basedperception filters may also store relevancy threshold information.Role-based perception filters may be created and updated manually or byusing machine learning algorithms.

During some investigations, it may not be readily apparent whatinvestigatory roles should participate in the investigation. Forexample, there may be objects of interest and relationships that arerelevant to a human trafficking expert, but that may escape the noticeof other types of investigators. Accordingly, in some embodiments, theelectronic processor 205 assigns relationship relevancy scores to therelationships for each of a plurality of investigatory roles. Whenrelevancy scores have been assigned, the electronic processor 205generates a role-based relationship alert for each of the plurality ofrelationships when the relationship relevancy score for the relationshipexceeds the relevancy threshold. In this way, users of the server 102are alerted that a particular investigatory role should be involved inthe investigation.

At block 310, the electronic processor 205 generates a graphicalrepresentation including a first indication of the subset of the objectsof interest and a second indication of the subset of the plurality ofrelationships. For example, as illustrated in FIG. 4, the graphicalrepresentation 400 is a graph that includes a first indication of thesubset of the objects of interest, represented by the circles 404, and asecond indication of the subset of the plurality of relationships,represented by the lines 406. In some embodiments, the electronicprocessor 205 generates the graphical representation using informationmapping techniques. For example, the graphical representation may be aconcept map, a knowledge map, a topic map, a link analysis graph, or thelike. The graphical representation may also be, for example, aforce-directed graph. In some embodiments, the electronic processor 205generates the graphical representation by superimposing the firstindication on images of its associated objects of interest within animage related to the investigation. For example, as shown in FIG. 4,some of the circles 404 are superimposed on an image 402 from theinvestigation. In the embodiment illustrated, the graphicalrepresentation 400 also includes a supplementary area 408. Thesupplementary area 408 is used to portray objects of interest that maynot be tangible (for example, data relating to the investigation), orthat may not have a corresponding image within the image related to theinvestigation.

At block 312, the electronic processor 205 presents the graphicalrepresentation on the display 220. In some embodiments, the electronicprocessor 205 transmits the graphical representation to a remote device(for example, the computing device 108) for display.

In some cases, embodiments of the system 100 and the method 300 may beused to identify missing objects of interest. For example, four out offive ingredients needed to make an illicit drug may be recovered duringan investigation. Identifying the missing ingredient can notifyinvestigators what to look for during subsequent investigation.Accordingly, in some embodiments, the electronic processor 205 mayidentify a second subset of the plurality of objects of interest. Forexample, metadata for the second subset of objects may indicate thatthey are ingredients used to make a particular illicit drug. In someembodiments, the electronic processor 205 compares the second subset topredetermined combinations of objects, each combination representing theingredient set for a different illicit drug. When the second subsetpartially matches a predetermined combination, the electronic processor205 can determine, based on the partial match, the missing ingredient oringredients. In such embodiments, the graphical representation includesa third indication of the missing object of interest and a fourthindication of a relationship between the missing object of interest andat least one of the objects of interest in the second subset. In someembodiments, the third and fourth indications are highlighted (forexample, by using a different color) to indicate that missing objects,which may be relevant to the investigation, have been identified.

As a user of the server 102 operates the system, he or she provides userinputs. For example, the user may identify one or more relationships asrelevant to a particular investigatory role. In another example, theuser interacts with the graphical representation, adding objects orrelationships, selecting existing objects and relationships, adding orchanging metadata, and the like. In some embodiments, the electronicprocessor 205 determines a user investigatory role for the user, forexample, by retrieving a user profile for the user. In some embodiments,the user identifies his or her investigatory role to the server 102 viaa graphical user interface. In some embodiments, the electronicprocessor 205 receives the user inputs and updates the role-basedperception filter corresponding to the user role for the user based onthe user input. In some embodiments, one or more classifiers associatedwith the role of the user are updated based on the actions and inputs ofthe user.

FIG. 5 illustrates an example method 500 for filtering a plurality ofobjects of interest associated with an incident scene. The method 500 isdescribed as being performed by the server 102 and, in particular, theelectronic processor 205. However, it should be understood that in someembodiments, portions of the method 500 may be performed external to theserver 102 by other devices, including for example, the image capturedevice 106 and the computing device 108.

At block 502, the electronic processor 205 receives a plurality ofobject identifiers. Each of the object identifiers corresponds to one ofthe plurality of objects of interest associated with the image of theincident scene.

At block 504, the electronic processor 205 receives metadata for theplurality of object identifiers. At block 506, the electronic processor205 determines a plurality of relationships based on the metadata, whereeach of the plurality of relationships corresponds to at least two ofthe plurality of objects of interest. Determining relationships isdescribed above with respect to the method 300.

At block 508, the electronic processor 205 generates a graphicalrepresentation based on the plurality of objects of interest and theplurality of relationships. FIG. 6 illustrates an example graphicalrepresentation 600 including an image 602 of the incident scene. Thegraphical representation 600 includes a plurality of nodes (representedby circles, for example, node 604). Each node providing an indication ofone of the plurality of objects of interest. The graphicalrepresentation 600 also includes a plurality of edges (represented bylines, for example, edge 606). Each edge connects two of the pluralityof nodes and provides an indication of one of the plurality ofrelationships. In the example illustrated, the nodes are superimposed onthe image on or near the images of the objects of interest that theyrepresent in the image 602 of the incident scene. As with the graphicalrepresentation 400 in FIG. 4, the graphical representation 600 includesa supplementary area 608, where intangible objects of interest orobjects of interest lacking images in the image 602 can be represented.

Returning to FIG. 5, at block 510, the electronic processor 205 receivesa role identifier. As described above with respect to the method 300,the role identifier identifies an investigatory role. In someembodiments, more than one role identifier is received.

As shown FIG. 6, a large quantity of nodes and edges can make itdifficult to discern information from the graphical representation 600.Accordingly, at block 512, in response to receiving the role identifier,the electronic processor 205 applies a role-based perception filtercorresponding to the role identifier to the plurality of relationships(and, thus, the plurality of objects) to generate a filtered graphicalrepresentation. The electronic processor 205 uses the role-basedperception filter, described above, to filter out relationships relevantto the role corresponding to the role-based perception filter. In oneexample, illustrated in FIG. 4, the nodes and edges that do notcorrespond to the role are removed from the graphical representation 600to produce the filtered graphical representation 400. In someembodiments, the nodes and edges corresponding to the role arehighlighted, for example, using a different color.

At block 514, the electronic processor 205 presents the filteredgraphical representation on a display (for example, the display 220).

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has,”“having,” “includes,” “including,” “contains,” “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a,” “has . . . a,” “includes . . . a,” or “contains . .. a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially,” “essentially,”“approximately,” “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

We claim:
 1. A system for filtering a plurality of objects of interestassociated with an investigation, the system comprising: a database, adisplay, an electronic processor communicatively coupled to the databaseand the display, and configured to receive a role identifier; retrieve,from the database, metadata corresponding to the plurality of objects ofinterest; determine, with a classifier, a plurality of relationshipsbetween at least two of the objects of interest based on the metadata;identify, based on the metadata, a subset of the plurality ofrelationships that are associated with the role identifier, the subsetof the plurality of relationships including a subset of the plurality ofobjects of interest; generate a graphical representation including afirst indication of the subset of the objects of interest and a secondindication of the subset of the plurality of relationships; and presentthe graphical representation on the display.
 2. The system of claim 1,wherein the metadata includes at least one selected from the groupconsisting of a timestamp, a location, an image of the object, an objectidentifier, an investigation identifier, an incident type, a potentialrelationship to another object of interest, and an object type.
 3. Thesystem of claim 1, wherein the electronic processor is furtherconfigured to assign to each of the plurality of relationships arelationship relevancy score based on the metadata for the objects ofinterest related by the relationship; and identify the subset of therelationships based further on the relationship relevancy score.
 4. Thesystem of claim 3, wherein the electronic processor is furtherconfigured to, for each of the plurality of relationships, assign aweight to the relationship relevancy score to generate a weightedrelationship relevancy score.
 5. The system of claim 1, wherein theelectronic processor is further configured to for each of a plurality ofrole identifiers assign to each of the plurality of relationships arelationship relevancy score based on the role identifier and themetadata for the objects of interest related by the relationship; andfor each of the plurality of relationships, generate a relationshipalert when the relationship relevancy score for the relationship exceedsa relevancy threshold.
 6. The system of claim 1, wherein the electronicprocessor is further configured to determine at least one missing objectof interest when a second subset of the plurality of objects of interestpartially matches a predetermined combination of objects related to thesecond subset, wherein the graphical representation includes a thirdindication of the missing object of interest and a fourth indication ofa relationship between the missing object of interest and at least oneof the objects of interest in the second subset.
 7. The system of claim1, wherein the electronic processor is further configured to generate agraphical representation by superimposing the first indication on imagesof its associated objects of interest within an image related to theinvestigation.
 8. The system of claim 1, wherein the electronicprocessor is further configured to determine a user role for a user;receive a user input from the user corresponding to an interaction withthe graphical representation; and update a role-based perception filtercorresponding to the user role based on the user input.
 9. The system ofclaim 1, wherein the graphical representation is one selected from thegroup consisting of a force-directed graph and a link analysis graph.10. A method for filtering a plurality of objects of interest associatedwith an investigation, the method comprising: receiving, with anelectronic processor, a role identifier; retrieving, from a databasecommunicatively coupled to the electronic processor, metadatacorresponding to the plurality of objects of interest; determining, witha classifier, a plurality of relationships between at least two of theobjects of interest based on the metadata; identifying, based on themetadata, a subset of the plurality of relationships that are associatedwith the role identifier, the subset of the plurality of relationshipsincluding a subset of the plurality of objects of interest; generating agraphical representation including a first indication of the subset ofthe objects of interest and a second indication of the subset of theplurality of relationships; and presenting the graphical representationon a display communicatively coupled to the electronic processor. 11.The method of claim 10, wherein retrieving metadata includes retrievingat least one selected from the group consisting of a timestamp, alocation, an image of the object, an object identifier, an investigationidentifier, an incident type, a role identifier, a potentialrelationship to another object of interest, and an object type.
 12. Themethod of claim 10, further comprising: assigning to each of theplurality of relationships a relationship relevancy score based on themetadata for the objects of interest related by the relationship; andidentifying the subset of the plurality of relationships based furtheron the relationship relevancy score.
 13. The method of claim 12, furthercomprising: for each of the plurality of relationships, assigning aweight to the relationship relevancy score to generate a weightedrelationship relevancy score.
 14. The method of claim 10, furthercomprising: for each of a plurality of role identifiers assigning toeach of the plurality of relationships a relationship relevancy scorebased on the role identifier and the metadata; and for each of theplurality of relationships, generating a relationship alert when therelationship relevancy score for the relationship exceeds a relevancythreshold.
 15. The method of claim 10, further comprising: determiningat least one missing object of interest when a second subset of theplurality of objects of interest partially matches a predeterminedcombination of objects related to the second subset, wherein generatingthe graphical representation includes generating a third indication ofthe missing object of interest and a fourth indication of a relationshipbetween the missing object of interest and at least one of the objectsof interest in the second subset.
 16. The method of claim 10, furthercomprising: determining a user role for a user; receiving a user inputfrom the user corresponding to an interaction with the graphicalrepresentation; and updating a role-based perception filtercorresponding to the user role based on the user input.
 17. The methodof claim 10, wherein generating the graphical representation includesgenerating one selected from the group consisting of a force-directedgraph and a link analysis graph.
 18. A method for filtering a pluralityof objects of interest, the method comprising: receiving, with anelectronic processor, a plurality of object identifiers, each of theobject identifiers corresponding to one of the plurality of objects ofinterest associated with an image of an incident scene; receivingmetadata for the plurality of object identifiers; determining, with theelectronic processor, a plurality of relationships based on themetadata, where each of the plurality of relationships corresponds to atleast two of the plurality of objects of interest; generating agraphical representation based on the plurality of objects of interestand the plurality of relationships, the graphical representationincluding a plurality of nodes, each node providing an indication of oneof the plurality of objects of interest, and a plurality of edges, eachedge connecting two of the plurality of nodes and providing anindication of one of the plurality of relationships; receiving a roleidentifier; in response to receiving the role identifier, applying arole-based perception filter corresponding to the role identifier to theplurality of objects of interest and the plurality of relationships togenerate a filtered graphical representation; and presenting thefiltered graphical representation on a display.